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Abstract. This study describes a research 
focused on science teachers’ evaluation of 
natural science literacy of research and com- 
prehension competence in their students. 
Natural science literacy of research and 
comprehension competence is defined as an 
essential part of science literacy — as a com- 
petence to find, evaluate and use science 
knowledge stored on the Internet (as a part 
of extended memory) to solve a problem in 
a science class and to construct students’ 
own science knowledge. Online science 
literacy was defined in terms of the following 
aspects: basic skills (which include computer 
basics, web searching basics, and general 
navigation basics), locating information, 
finding a suitable website, locating the infor- 
mation on the website, critically evaluating 
the information according to its reliability 
and according to its relevance for the science 
class assessment. The data were collected 
through a 53-item Likert - scale question- 
naire. The items were adopted from the TICA 
questionnaire for assessing students’ general 
online reading competence. Science teach- 
ers from 5 different levels of pre-university 
education assessed their students’ online 
science literacy in order to evaluate their 
students’ competence to use the Internet 

as a storage and as a source of knowledge 
for teaching/learning process in the science 
class, to re-evaluate their online teaching 
practice and the need for implementation 
of natural science literacy of research and 
comprehension competence in their science 
curriculum. 
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Introduction 


The process of shifting reading from page to screen was accompanied 
with the fact, “old” literacies were not suitable for reading and learning in 
new digital environments any more - users of online information sources 
and learning materials began to develop a new kind of literacy — so called 
new literacy, or, to be more precise: new literacies. 

The term “new literacies” was introduced by Gallego and Hollingsworth 
(1992). In more than two decades the interpretation of the meaning of the 
term changed several times. The current perspective suggests that new lit- 
eracy is rapidly changing and transforming — a parallel process to emerging 
new communication and information technologies and to changing social 
practices (Abersek, Borstner, & Bregant, 2014, Abersek, 2013). With each such 
change new skills are required to make use of new technologies. (Baker, 2010; 
Gee, 2007).“Moreover, with the Internet, literacy is not just new today; it is new 
every day, as additional technologies for literacy regularly and rapidly appear 
online” (Leu, 2014, p. 2). This changing nature of new literacies confronts the 
theory with the serious problem: how to describe the object (new literacy 
competence), if the object is permanently changing? 

Recently, a dual level theory of New Literacies has been proposed to 
respond to this problem (Leu, Kinzer, Coiro, Castek, & Henry, 2013). It observes 
new literacies on two levels: the uppercase and the lowercase new literacies. 
Uppercase new literacies research is focused on new social practices Internet 
makes possible with technologies such as instant messaging, social networks, 
blogs, wikis, and e-mail (Greenhow, Robelia & Hughes 2009). Lowercase new 
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literacy theories explore a specific area of new literacies and/or a new technology needed for social communicative 
transactions. One of the lowercase theories is exploring the field of new literacies of online research and compre- 
hension (the term replaced the term “online reading comprehension”). The new literacies of online research and 
comprehension seek to describe what happens when we read online to learn. 


Theoretical Framework 


First theories tried to explain the nature of reading in Internet contexts and required the ability to flexibly reas- 
semble existing knowledge with new knowledge applications customized to each new reading situation (Spiro, 
2004). A cognitive flexibility theory claimed that older notions of knowledge domains used to interpret and predict the 
meaning of printed text (Anderson, 1994) no longer sufficiently explain the knowledge domains required of readers in 
Web-based contexts and argued (Spiro, 2004) that learning strategies working in simple domains are exactly opposite 
of those best for dealing with complex domains such as the Internet. 

Later research rejected this theory. Reading was confirmed as an active, constructive, meaning-making process, 
in which readers actively construct meaning, as they interact with. Expert readers use a range of strategic cognitive 
processes to select, organize, connect, and evaluate what they read. These strategies include asking questions, develop- 
ing connections (Jesensek, 2011), and making inferences (Pressley & Afflerbach, 1995). In addition, readers use their 
existing knowledge to more clearly understand ideas and information in texts, make predictions about what might 
come next, and reason strategically when they have difficulties in the process of comprehension. Use of informational 
texts requires readers to attend to structural text features, and evaluate the relevancy of the text in relation to the task 
(Dreher, 2002; Goldman & Rakestraw, 2000). 

According to the new literacies of online research and comprehension, these strategies play an important role in 
online reading comprehension too. But they are almost not sufficient for successful searching for information and for 
using the Internet as a source of knowledge in the ICT supported science class, since Internet text differs from linear 
text in many ways (Dolenc, & Abersek, 2015). Internet is an open network system “a kind of informational environ- 
ment in which textual materials and ideas are linked to one another in multiple ways” (Burbules & Callister, 2000, p. 
43). Links embedded within hypertext systems are constructed so that readers must select a target location in order to 
move through the text. Moreover, hypertext makes explicit and external a range of possible interconnections between 
texts and guides the conception of readers who can not only follow embedded connections created by the author but 
also construct their own personal pathways through multiple texts. The Internet “text”/site usually contains outside 
advertisements, links that change from one day to the next, or pathways to information that are completely outside the 
realm of its intended purpose. Internet texts are part of a complex open-ended information system that changes daily in 
structure, form, and content. They offer distracting advertisements, inconsistent text structures, broken links, and access 
to an infinite amount of information completely unrelated to their intended reading purpose. And last, but not unimport- 
ant: Internet texts are combined in complex ways (Coiro & Dobler, 2007) and are often intermingled with hidden social, 
economic, and political messages (Leu & Kinzer, 2000). 

New school agendas all over the world recommend Internet “text” as a knowledge source and as a source of 
information — and natural science didactics seem to be more open to the concept than social science didactics, which 
remains far more attached to traditional linear sources of knowledge. And all that without considering the open ques- 
tion of prerequisite for such shift from page to screen and from linear to networked text structure — the new literacy 
of online research and comprehension. 

Current (qualitative) research (Coiro, 2007; 2011; Leu et al., 2008) brought light to metacognitive processes 
and inferential reasoning processes expert e-readers are using by their successful reading of e-texts, explained 
the role of pre-knowledge in this process and why they contribute to better comprehension online. According to 
these findings, the new literacy of online research and comprehension is structured and contains in the frame of each 
structural element skills, very similar to those particularly useful in the process of linear reading, and additional 
complexities needed for Internet comprehension. 
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Table 1. Similarities and differences between linear and Internet text readings (Adopted after Coiro & Dobler, 


2007). 
senand bolp iis .; Additional complexities, needed in open 
comprehension Similarities between linear and open hypertext reading 
; hypertext comprehension processes 
strategies 
Pre knowledge — pre knowledge of the topic — prior knowledge of hypertext structure / 
e knowledge website structure; 
° misconceptions ~ prior knowledge of Web-based search 
- engines - basic skills 
e vocabulary (general, specific) 
e computer basics, 
— pre knowledge of printed informational text structures ¢ navigational basics 
e Web searching basics 
Inferential reasoning — creating coherence: — forward inferential reasoning; 
e text based coherence — multilayered reading process across 
° general representation (situation model) hypertext structure and three dimensional 
— inferential reasoning strategies: Internet spaces 
e literal matching skills 
e structural cues 
e context clues 
Metacognitive/ — conventional metacognitive strategies for comprehension — self-regulated recursive circle intertwined 
self-regulated processes monitoring and repair; with physical reading actions (typing, click- 
— connected components of a larger strategic reading ing, scrolling, dragging); 
process; — rapid information-seeking cycles within 
— self regulated recursive circle extremely short text passages 


Inferential reasoning is a central component of skilled reading (Garnham & Oakhill, 1996). It is the ability to read 
between the lines while making connections not explicitly stated in the text. Readers with sufficient prior knowledge, 
make more inferences than less knowledgeable readers in order to facilitate their comprehension of informational 
text. 

Let us examine the process of inferential reasoning more closely. The most recognized theory that explains 
this topic is a C-/ model of text comprehension (Kintsch, 1988; van Dijk & Kintsch, 1978). Although Kintsch model 
has been developed in the 1980s, it is still used for predicting the comprehension of text based knowledge - also 
on the Internet. In the Kintsch model, the reader's memory of the text is represented at 3 levels: representation of 
the words and sentences, the meaning of the text (textbase), and a general representation of what is described 
by the text incorporating outside background knowledge (situation model). In the frame of explanation text, the 
typical textform in science education, which is based on facts and used to explain the sequence, cause and effect 
of an event, the information, called propositions, are connected. Such connections are called semantic coherence 
relations. 

Propositions that have overlapping arguments are semantically related and create coherence. In cases, when 
the proposition, processed in the particular moment, does not share arguments with propositions in short-term 
memory, the reader must create a bridging inference — with the aim to maintain coherence (Kintsch & van Dijk, 
1978). For such creation of bridging inference, the reader must use his pre knowledge in order to fill in the missing 
information. 

This is the reason why for learning from the Internet a skilled learner needs additional inferential reasoning skills. 
Coiro and Dobler (2007) speak of specific complexities of inferential reasoning required in Internet contexts. In their experiment 
they observed two types of specific inferential reasoning in the process of learning from Internet: forward inferential reasoning and 
a multilayered inferential reasoning process. They report “the nature of Internet text appeared to prompt a high incidence 
of forward inferential reasoning (e.g., predictions) beyond the level typically involved in the comprehension of printed 
informational text” (Coiro & Dobler, p. 233). They found evidence to suggest that skilled readers in our study appeared 
to make forward inferences (e.g., predictions) within Internet text each time they were confronted with one or more 
hyperlinks on a given page. Also, they observed skilled readers, combining their use of traditionally conceived inferential 
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reasoning strategies with a new understanding that the relevant information may be “hidden” beneath several layers 
of links on a website as opposed to within one visible layer of information in a printed book. Comprehending Internet 
texts encourage readers to anticipate their understanding through multiple layers that are almost always hidden from 
view, demands many more attempts to infer, predict, and evaluate reading choices (e.g., hyperlinks followed) while 
anticipating the relevance of information in an open information space multiple levels beyond a visible link. 

Successful learning from Internet text is essentially linked with metacognitive strategies for comprehen- 
sion monitoring and repair. As predicted, conventional metacognitive reasoning and additional metacognitive 
complexities contribute to find and process the information and to construct coherent knowledge. Among these 
additional metacognitive complexities is a self-regulated recursive circle connected with physical reading actions 
such as typing, clicking, scrolling and dragging. Every time a reader is moving the mouse, a circle is initiated: plan- 
ning, predicting, monitoring, evaluating. And, parallel to the next move, a regulated recursive circle is generated 
again and again. Searching and learning process on the Internet is conditioned with the competence of using 
these rapid information-seeking circles within extremely short text passages. 

As in conventional linear literacy pre knowledge is a prerequisite for inferential reasoning and metacognitive 
processes in reading and learning from Internet texts. In linear text the author creates a line of coherent arguments 
through the text, then makes words and sentences flow together through common referents. This helps the reader 
in structuring the information in the text to fit into the knowledge structures of what has been read previously. 
Differently, Internet text does not help the reader with the carefully selected order of information. This, as Foltz 
(1996) points out, is the main problem connected with hypertext comprehension on the Internet. It is essential 
for the hypertext structure that in any text section in a hypertext, there are usually a variety of other sections to 
which a reader can jump. This freedom of choice can cause difficulties in comprehension process concerning 
finding the coherence. In the reading process of hypertext it is impossible to predict what the reader will read 
first and what he is going to read next. Consequently, it is impossible to equip the reader with the information 
he would have needed in the short term memory to find the coherence with the information he is reading at the 
contemporary moment. It is not possible for the author of the Internet text to anticipate all the possible places to 
which a reader may jump and therefore, it is also not possible to maintain good macro coherence for all possible 
links. Incoherent jumps result in additional processing load for the reader. In such cases the reader generates the 
necessary inferences to incorporate the textual information from the new node into what has been previously read. 
The consequence of variety of the possible links in a hypertext is the smaller amount of propositional overlap. This 
corresponds to a smaller amount of coherence of the text reception on the reader’s side and may cause difficulty 
in the reader's comprehension — simply because the reader must create more bridging inferences. If readers don't 
have the proper pre knowledge, these inferences cause an additional mental workload for the reader. The consequence 
of this is lower comprehension. 

But only pre knowledge of the topic and vocabulary (Koletnik, 2013) is not sufficient for successful compre- 
hending of the Internet text. According to Coiro & Dobler (2007), there are two additional sets of pre knowledge: 
pre knowledge of informational website structures and pre knowledge of Web based search engines. 

Pre knowledge of informational website structures to guide their reading on the Internet is essential in the process 
of navigating a web site, for recognizing and negotiating hierarchical and non-linear hyperlinks, navigational icons, 
interactive multimedia, and browser toolbars (Bilal, 2000; Eagleton, 2003). Every time a user opens a new website, 
he needs time to find out the internal logic of its structure, learns to recognize visual signals for its linear and nonlin- 
ear links, to find out what a particular navigation icon means, to see which multimedia connections are offered for 
additional information on the topic and — not unimportant X how the interruptions (advertisements, non-relevant 
links) are marked. The navigation will be easier the next time the user opens this website. He will remember, what 
he has learned about its structure, and this knowledge will help him to find the needed information more quickly. 
After longer experience with different websites, the user will be able to adapt very quickly to the structure of any new 
websites, since they are usually structured according to some general rules. Coiro and Dobler research (2007) reviled, 
skilled Internet readers possess important sources of knowledge about the structure and organization of informational 
websites that forms the decisions they make during online comprehension. 

Pre knowledge of Web based search engines involves understanding the processes of browsing, selecting appropri- 
ate search engines, formulating keyword searches, negotiating subject hierarchies, and evaluating annotated search 
results (Bilal, 2000). A closer look at this knowledge shows three areas where this knowledge is stored. The first includes 
the so-called computer basics, such as turning the engine on/off, following the cursor with the eyes, using the mouse, 
logging in/out, using a word processor, creating a new file. The second includes basic knowledge for finding informa- 
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tion on the Internet: how to formulate the key words, where to write them on the web site, how to narrow or widen 
the search process. And the third includes knowledge on how to navigate in open hypertext on the World Wide Web 
and how to evaluate what has been found - which criteria to use to decide whether the information is reliable (and 
can be used for school work) or not. 

Based on Coiro and Dobler findings (2007), in later research Leu et al. (2013) suggested that at least five pro- 
cessing practices occur during online research and comprehension: a) reading to define important questions; b) 
reading to locate online information; c) reading to critically evaluate online information; d) reading to synthesize 
online information, and e) reading and writing to communicate online information. 

Online reading comprehension always begins with a question or a problem. This is an important source in the 
differences between online and offline reading comprehension. It is known that the new literacies of online read- 
ing comprehension occur within a process that includes the skills and strategies required to identify an important 
question directing the reader to locate, critically evaluate, synthesize, and communicate information with the 
Internet (Leu et al., 2008). Users read on the Internet to solve problems and answer questions. How a problem is 
framed or how a question is understood is a central aspect of online reading comprehension. 

Locating information online is another aspect of online reading comprehension. It also requires new online 
reading comprehension skills such as using a search engine, reading search engine results, or quickly reading a 
web page to locate the best link to the information that is required. Many students lack these skills. Of those who 
do use a Search engine, for example, many do not appear to know how to read search engine results, instead they 
click down the list of links in a “click and look” strategy (Leu et al., 2008). Locating information during the online 
reading comprehension process may create a bottleneck for the subsequent skills of online reading comprehension 
(Henry, 2007). That is, those who possess the online reading comprehension skills necessary to locate information 
can continue to read and solve their problem; those who do not possess these skills cannot. In fact, this bottleneck 
may contribute to the lack of isomorphic performance between online and offline readers. 

Another area in which online reading comprehension requires a unique set of skills is during critical evalu- 
ation. Whereas critical evaluation is important when reading offline information, it is perhaps more important 
online, where anyone can publish anything; knowing the stance and bias of an author becomes paramount to 
comprehension and learning. Determining this in online contexts requires new comprehension skills and strate- 
gies (Bregant, Stozer, & Cerkvenik, 2010). 

Successful online research and comprehension also requires the ability to read and to synthesize online 
information from multiple online sources. Synthesis or integration of separate and unique ideas is recognized as 
the most demanding part of the offline comprehension process. In reading on the Internet it becomes even more 
demanding. The Internet introduces additional challenges for coordinating and synthesizing enormous amounts 
of information presented in multiple media and in multiple media formats from nearly unlimited sources (Leu et 
al., 2014). 

A final component of successful online research and comprehension is the ability of reading and writing to 
communicate online information via Internet, while interacting with others to seek for more information or to share 
what you have learned (Leu et al., 2014). 

In summing up all this theoretical background, we can repeat, what we have already mentioned: expert readers 
use, for reading on the Internet, a range of strategic cognitive processes to select, organize, connect, and evaluate what 
they read. These strategies include asking questions, developing connections, and making inferences, evaluating and 
synthesizing, what they have found, into a coherent knowledge about the topic and/or the research question. 


The Aim of the Research 


Theoretical background brings us to the central question of the study, which was implemented in spring 2015 
in Slovenia: to what extent are students in compulsory and secondary education prepared for the shift from “page to 
screen’, or, to be more precise, are they competent for online research and comprehension and what is the level of 
their new natural science online research and comprehension literacy. Therefore, the study examined compulsory 
and secondary school natural science teachers’ assessment of their students’ new natural science literacies of online 
research and comprehension competence. 
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Research Methodology 
General Background of Research 


The central question of the research was to find out: are students, who all belong to a“Z generation’, the so 
called “digital natives”, online research and comprehension competent. Are they all prepared for a total switch to 
online learning and to exclusively ICT supported natural science teaching paradigm? This question is very important, 
since new school agendas all over the world recommend Internet “text” as a knowledge source and as a source of 
information — and natural science didactics seem to be more open to the concept than social science didactics. In this 
context the opinions on the competence in new literacies of online research and comprehension are strongly divided: 
some were convinced, the Z generation is totally competent for Internet communication, also for gaining and sharing 
new knowledge, and others (more detailed research), who warn, the digital natives, at least some of them, have in 
the process of learning online remarkable difficulties, which will greatly decrease their learning results, if the switch 
from conventional to ICT teaching paradigm will happen too soon and without differentiation according to the level 
of their online research and comprehension literacy competences. 

Four research questions were examined to reveal the answer to the central question of the study: 1. How do 
natural science teachers assess their students’ competence of basic online skills (computer basics, web searching 
basic, navigation basics)? 2. How do natural science teachers assess their students’ competence for locating infor- 
mation by using a search engine? 3. How do natural science teachers assess their students’ competence of web 
site navigation? 4. How do natural science teachers assess their students’ competence of evaluation of research 
results? 


Sample Selection 


Asample comprised 70 elementary and natural science teachers, 48 of them were compulsory school teachers 
and 22 upper secondary teachers. In the Slovene school system compulsory school (9 years) is divided into 3 periods, 
called trienniums. In the first triennium (age 6 — 8) and in the first two years of 2"? triennium (age 9 — 11) natural 
science is taught by elementary teachers in the last year of 2"? triennium and in the 3 triennium (12 - 14), natural 
science is taught by biology, physics, chemistry, geography teachers and teachers for science and technology. At 
the upper secondary level natural science is taught by subject specialists. The sample of (48) compulsory teachers 
was divided into three groups according to triennium, where they teach. This part of the sample consisted of 22 
natural science teachers from the gymnasium upper secondary program. Such sample was selected because: it 
covered all groups of pre university education from the age of 6, where it was examined, the Z generation really 
enters the school system online research and comprehension literacy and consequently does not need teachers 
assistance in this matter, to the secondary education, where the intent was to find out if all members of Z genera- 
tion develop online research and comprehension literacy in the present form of pre university education. 


Instrument and Procedures 


Data were collected through 53 - item Likert-scale questionnaire. The items were adopted from TICA Check- 
list, developed in Teaching Internet Comprehension Skills for Adolescents project, which was focused on studying 
skills, essential to online reading comprehension (Leu et al., 2008). The original TICA checklist of online reading 
comprehension contained items from 5 areas, required during online reading comprehension: understanding 
and developing questions, locating information, critically evaluating information, synthesizing the information 
and communicating the information. The adaptation, used for our research, focused on computer basics (Appen- 
dix A in the original survey) and on locating information by using a search engine (finding the useful web site), 
navigating a web site and evaluating the results of research (Appendix B in the original survey). Also the number 
of items was reduced - in the checklist for computer basics from 29 to 21, in the checklist for locating information 
by using a search engine from 26 to 15, in the checklist for navigating a web site from 8 to 7 and in the checklist 
evaluating the results of research (evaluation and reliability) from 16 to 10. The survey, used in the research did not 
contain questions about the competence for generating questions, synthesizing information and communicating 
information. 

The questionnaire was handed out to groups of teachers, above defined. In the four compulsory schools, 
where data were selected, an assistant researcher handed out the questionnaires personally. A short motivational 
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conversation followed: the purpose of this conversation was to motivate teachers to pay attention to the ques- 
tionnaires and to focus on the topic for the whole duration of answering the items in question. At the secondary 
level, the school headmasters were asked to hand out the questionnaires at the pedagogical conference, where 
all teachers were gathered, and to motivate them to focus on the questionnaire. On this occasion the headmaster 
stated that the results of the research will be valuable for the next teaching practice. 

The questionnaire for teachers had a following introduction: /n a contemporary school it was declared, memo- 
rizing knowledge is not needed anymore, the so called knowledge in the digital world is available on the Internet and 
always “just a click away”. In this context, one should focus on the questions: “Are all our students competent to use 
Internet as a learning source, and for storing knowledge?” Are they really competent to find the information/knowledge 
at the moment, when it is needed? Or in short: 1S IT TRUE THAT THE GENERATION X ENTERS THE SCHOOL AS DIGITALLY/ 
ONLINE LITERATE? |n the following text teachers were asked to evaluate online literacy of their students. Elementary 
teachers’ questionnaire included also the guideline that they should evaluate their students’ online literacy in the 
teaching situation in the natural science class. 


Data Analysis 


Quantitative data of elementary teachers and natural science teachers were collected. After verifying that 
data were free of errors, quantitative analyses were conducted and analyzed according to the following phases, 
or by: encoding, defining and organizing the data and interpreting the results. For statistic processing of the data 
an IBM SPSS program was used. For the basic statistic interpretation of the results mean and standard deviation 
values of the data were used. 


Results of Research 


Table 2 shows mean values and standard deviation values of natural science teachers’and elementary teachers’ 
evaluation of their students’ online basic skills, computer basic skills, web searching basic skills and their general 
navigation basic skills. 

In general, upper secondary teachers evaluated that their students were quite proficient in all 26 computer 
basic skills, listed on the checklist for computer basics. Only two items scored lower than M=4.50: item “open pro- 
grams and files using icons and/or the Start Menu (PC)” scored 4.36 (S.D.=0.83) and item “Open a new tab” scored 
4.23 (SD=0.83). All other items scored between 4.50 (SD=0.95) and 4.95 (SD=0.97). On the other side, teachers of 1* 
triennium evaluated the skills of their students’ computer basics as very low: if we frame out the result that almost 
all students can turn on the computer and use the mouse/track (M=4.41 and M=4.31) and the fact that almost all 
of them can follow the class/school rules for computer use (which is not strictly a computer basics), remains the 
fact that teachers evaluated all other components of computer basics skills between M=1.65 (SD=0.87) and M=2. 
88 (SD=0.82). 


Table 2. Check list 1- online basic skills. 
1st 2nd 3rd Gymnasium 


Mean SD Mean SD Mean SD Mean SD 
1. Computer Basics 


Turn a computer on/off 4.41 1.07 4.60 1.19 4.8 0.89 4.95 1.04 


Use the mouse/track pad 4.35 0.91 4.8 1.04 4.8 0.79 4.86 1.04 
Follow classroom and school rules for computer use 3.47 1.14 3.8 0.99 4.19 0.89 4.45 0.99 
Open programs and files using icons and/or the 

Start Menu (PC) 2.88 0.82 4.47 0.83 3.88 0.85 4.36 0.83 
Create/open a new folder/file 1.94 0.95 Bie 0.91 4.12 0.95 4.68 0.80 
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1st 2nd 3rd Gymnasium 


Mean SD Mean SD Mean SD Mean SD 


Launch a word processor 2.06 0.84 3.93 1.14 4.25 0.93 4.77 0.88 
Open a word processing file 1.88 0.97 2.87 0.82 4.12 1.05 4.77 0.87 
Type a short entry in a word processing file 2 0.83 o.2r 0.95 4.19 0.88 4.41 1 

Copy text 1.76 1.25 3.4 0.91 4.31 1.19 4.73 0.89 
Cut text 2.24 1.04 3.8 0.84 4.44 1.19 477 0.93 
Past text 1.82 1.01 330 0.97 4.88 1.04 4.77 1.06 
Name a word processing file and save it 1.71 0.97 3.6 0.83 4.63 0.99 4.77 0.84 
Open a new window 2.24 1,13 o00 1.23 4.25 0.83 4.73 0.97 
Open a new tab 1.65 0.87 2.8 1.04 3.56 0.80 4.23 0.83 


2. Web Searching Basics 


Locate and open a search engine 2 0.85 4.27 1.07 4.69 0.89 4.95 0.97 
ha words in the correct location of a search 271 0.91 467 0.91 469 0.85 489 113 
Use the refresh button 1.94 1.14 A 1.14 oN 0.95 45 0.87 
Use the “BACK” and “FORWARD” buttons 2.65 0.82 3.67 0.82 4.38 0.93 4.82 0.89 
3. General Navigation Basics 

Maximize/minimize windows 2.18 0.7 BNE 0.7 4.88 1.19 49 0.95 
Open and quit applications 2.16 0.81 3.93 0.81 4.5 1.05 4.95 0.93 
Toggle between windows 2 0.89 3.4 0.89 4.06 0.88 4.82 1.05 


The results of the Checklist 1 could be understood as the answer to the question: do the digital natives enter 
the school computer literate: The answer is: no, they do not, even more, almost all of them can turn on and off the 
computer and use the mouse/track pad, what they probably had learned using digital devices for play/fun, but 
a great majority of them doesn’t possess other 21 computer basics, they would have needed for searching and 
learning with the help of digital engines. 

Table 3 shows that elementary and natural science teachers evaluate their student’s competences for searching 
for a proper website for information and learning as remarkably lower than their computer basics competences. 


Table 3. Checklist 2 - Locating information by using a search engine /searching for a proper website. 
1st 2nd 3rd Gymnasium 
Mean SD Mean SD Mean SD Mean SD 


1. Locate at least one search engine. 2.16 1.06 Gio 0.89 4.88 0.93 477 0.83 


2. Use several of the following general search engine strategies during key word entry: 
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1st 2nd 3rd Gymnasium 


Mean SD Mean SD Mean SD Mean SD 


topic and focus 1.94 0.97 3.00 0.95 JO 0.91 4.09 0.88 
single and multiple key word entries 2.29 0.83 3.6 0.93 4.38 1.14 4.23 0.87 


phrases for key word entry 1.47 1.23 3.13 1.05 3.5 0.82 4.09 1 


3. Use several of the following more specialized search engine strategies during key word entry: 


quotation marks 1.24 1.01 2.00 1.19 2.00 0.91 3.41 0.87 
paraphrases and synonyms 1.24 0.97 3 1.19 2.56 1.09 3.41 0.89 
Boolean (and/or/nor) 1.18 0.93 Z07 1.01 2.44 0.7 bA0 0.79 


4. Read search engine results effectively to determine the most useful resource for a task using strategies such as: 


Knowing which portions of a search results page are 
sponsored; containing commercially placed links, 1.41 0.84 2.06 0.79 3.06 1.19 3.86 0.85 
and which are not. 


Skimming the main results before reading more 


1.24 0.97 200 0.89 3.06 1.19 3.82 1.04 
narrowly. 


Reading summaries carefully and inferring meaning 
in the search engine results page to determine the 1.41 1.43 2.2 0.85 2.38 1.01 3:00 1.04 
best possible site to visit. 


Understanding the meaning of bold face terms in 


1.35 0.87 2.87 0.95 2.94 0.91 3.00 0.99 
the results. 
Understanding the meaning of URLs in search 104 0.89 9 0.93 9 34 1.02 3.55 0.83 
results (.com, .org, .edu, .net). 
Knowing when the first item is not the best item for 1.65 0.79 26 1.07 9 63 0.93 294 0.80 
a question. 
Knowing how to use the history pull down menu. 1.24 0.89 2.4 0.91 2.63 0.79 3.64 0.88 
5. Bookmark a site and access it later. 1.35 0.93 2.6 1.14 2.63 0.89 3.23 0.87 


Upper secondary teachers expressed quite a great confidence in their students’ competence to locate at 
least one search engine (M=4.77; SD=0.83) similar as 3"? triennium teachers (M=4.88; SD=0.93), but observing the 
competence of using search engine strategies and specialized search engine strategies, the difference between 
gymnasium (age 14 - 18) and 3" triennium (age 11-13) is noticeably larger. And also, comparing the results in 
general search strategies (topic and focus, single and multiple key word entries, phrases for key word entry) and 
specialized search engine strategies (quotation marks, paraphrases and synonyms, Boolean ) shows, that many 
of gymnasium students in their teachers’ evaluation could not use the specialized ones. The difference between 
gymnasium students and 3 triennium students is even larger when compared to the competence of reading 
search engine results effectively. 3"? triennium students scored between M=2.31 (SD=1.02) and M=3.06 (SD=1.19) 
and gymnasium students between M=2.91 (SD=0.80) and M=3.86 (SD=0.85). Observing the results of the teachers’ 
evaluation of 1 and 2"? triennium students’ competence for location the information by searching for the proper 
website, it could be said that this competence is not developed enough for implementing Internet as a learning 
source in the natural science education in the first half of compulsory school - at least not as a frontal didactical 
tool for all students. 

Table 4 shows, that elementary and natural science teachers evaluate their students’ competences for navi- 
gating a website for information and learning lower than their computer basics competences and similar to their 
competence for searching for a proper website. 
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Table 4. Check list 3 - navigation on the web site. 


1st 2nd 3rd Gymnasium 

Mean SD Mean SD Mean SD Mean SD 
NAVIGATION on a Web site 
Quickly determine if a site is potentially useful and worth 1.06 0.87 9 53 0.95 3 113 3 AI 1.05 
a more careful reading 
Read the site more carefully to determine if the required 147 1 26 0.91 994 0.87 > 94 0.88 
information is located there. 
Predict information behind a link accurately to make 1.99 0.89 9 1.09 9 31 0.89 > 86 119 
efficient choices about where information is located. 
Use structural knowledge ofa web page to help locate 1.99 0.93 943 07 > 69 0.79 A5 119 
information, including the use of directories. 
eee Nee ee Oe 253 083 319 089 409 1.01 
information, without losing the initial web page. 
Know how to use an internal search engine to locate 1.59 0.81 29 103 > 88 0.83 377 0.97 
information on a site. 
Monitor the reading of a web page and know when it 147 0.89 9 33 104 3 123 3.36 113 


contains useful information and when it does not. 


Upper secondary teachers expressed high confidence that their gymnasium students’ knowledge of the 
text structure and the competence for using it in the process of navigating the website (M=4.50; SD=1.19), on the 
other side, 1* triennium teachers expressed that their students have no such knowledge - so they cannot use it for 
navigation of the site. Table 3 shows the dynamics of the development of this competence. Observing this dynamic 
shows that less than a half of the students between 12 and 14 can use knowledge of text structure for navigating 
the website. The lowest scored competence in the group of navigational competences was the competence to 
predict information behind a link, in order to make efficient choices where information is located (M=2.86; SD=1.19 
for gymnasium students). This means that even at the doorstep of the university, according to the opinion of their 
teachers, more than a half of students are not capable of multilayered inferential reasoning process, which is es- 
sential for searching and locating information on the Internet. 

Table 5 shows that elementary and natural science teachers evaluate their students’ competences for critically 
evaluating information in the process of learning significantly lower than their computer basics competences and 
slightly lower than their competence for navigating the chosen website. 

The competence for critical evaluation of information/knowledge, found on the website, is in the teachers’ 
opinion quite low. Even at the age 15 — 18 just little over a half of students can identify, evaluate, and recognize that 
all websites have an agenda, perspective, or bias and identify and evaluate bias, given a website with a clear bias 
(M=3.59). And the same amount of students know that Wikipedia is not a perfect portal of information when they 
search for information or knowledge for their natural science assignments. The lowest developed competence is 
the competence to use the fact that the broken links are a sign of an unreliable website (M=2.86). This can be ex- 
plained in connection with the absence of competence for using knowledge about the website structure, presented 
in Table 3. By observing the results of compulsory school students, it can be concluded, that the competence for 
evaluating what is reliable information and what not, is not sufficiently developed in order to use the Internet as 
an information source in natural science class, without the teachers’ didactical guidance. 
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Table 5. Check list 4 Evaluation of the information on the web site. 
1st 2nd 3rd Gymnasium 


Mean SD Mean SD Mean SD Mean SD 


Critically Evaluate Information 


Identify, evaluate, and recognize that all websites have 
an agenda, perspective, or bias and identify and evaluate 1.59 0.91 2.00 0.83 3.19 1:13 3.59 0.83 
bias, given a website with a clear bias. 


Identify and evaluate the author of a website whenever 


aan .; 1.05 1.09 1.87 1.23 2.31 0.87 2.86 0.80 
visiting an important new site. 


Identify several markers that may affect reliability, such as: 


e ls this a commercial site? 1.29 0.81 1.87 1.01 2.44 0.79 3.14 0.87 
: eae source (@.9., professor, 47, ggg 473 097 244 0.89 3 i 

e Does the website have links that are broken? 1.00 0.97 1.6 0.95 2.94 0.85 2.86 0.89 
e Does the information make sense? 1.59 0.78 1.87 0.93 20 0.95 3.18 0.93 
° aaa author include links to other reliable web- 1.29 1414 1.87 1.05 95 0.93 3.44 1.06 
e Does the website contain numerous typos? 1.35 0.82 1.47 0.88 4.19 1.05 3.05 0.84 
e Do the images or videos appear to be altered? 1.41 0.95 1.8 1.19 2.81 0.97 3.36 0.97 
Understand that Wikipedia is a reasonable, but imperfect, 153 0.91 26 119 2 81 113 3.59 0.83 


portal of information 


The results, presented in tables 2 — 5, answer the central question of the presented research: Are students, who 
all belong to a“Z generation’, the so called “digital natives’, online research and comprehension competent? Are 
they all prepared for a total switch to online learning and to an exclusively ICT supported natural science teaching 
paradigm? The results of the present research reveal that the X generation, generation of digital natives, does not 
enter school as digitally literate, and above all, that it does not have the new natural science literacies of online 
research and comprehension competence. This competence is a very structured competence, and the particular 
elements have, as the present research shows, different dynamics. If we can say, that the basic computer skills are 
sufficiently developed during the pre-university education, we cannot oversee, that competences searching for a 
proper website, navigating the website and critically evaluating the results found on the web site develop much later. At 
the beginning of the compulsory school they are practically nonexistent and they develop slowly during the next 
13 years — in just slightly more than half of the population. After realizing this, a following conclusion should be 
considered: natural science literacies of online research and comprehension competence should be methodically 
developed and should become a part of general and special natural science curricula. 


Discussion 


The results of the study must be considered in the context of several studies, which have investigated and 
compared the effectiveness of online learning/reading and offline learning — and their results, which have almost 
as arule shown the difficulties weak students will have, by switching from offline to online too soon — before they 
are ready. Hill and Hannafin (1997) have explored the cognitive strategies used by adult learners on the Internet and 
found that metacognitive strategies, prior knowledge of subject and Internet text systems, and perceived self-efficacy 
influenced their ability to interact with and learn from Internet text. Others have explored the nature of Internet 
search strategies among students in K-12 classrooms and found numerous obstacles in information seeking with 
open Internet text environments. Readers on the Internet experienced challenges associated with ineffective and 
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inefficient search processes (Bilal, 2000; Eagleton, 2003), cognitive overload and disorientation (DeStefano, & LeFe- 
vre, 2007), a tendency to drift from one search question to another (Lyons, Hoffman, Krajcik, & Soloway, 1997), and 
an inability to know how to use the information once it has been located (Wallace, Kupperman, Krajcik, & Soloway, 
2000, Coiro & Dobler, 2007)). Also, A. Mangens’ research of learning results gained by reading linear and online text 
showed significantly weaker results when students tried to gain knowledge from screen, even more, if studying time 
was limited (Mangen, 2012). She explained such results in relation to issues with navigation within the document 
and the necessary scrolling, which is inevitable when reading longer documents. Similarly DeStefano and LeFevre 
(2007) point out that hypertext confronts students with an additional cognitive load — and underline, readers with 
high prior knowledge are typically unaffected when the text structure was nonlinear and had many imbedded links. 
On the other hand, students with weaker knowledge needed the guidance to comprehend the information delivered 
by the information of the hierarchical structure of the text. DeStefano and LeFevre concluded their research with a 
warning: students with lesser abilities need guidance in learning from hypertext, especially when hypertext is not 
optimally structured, when the choices are totally free (if they are not limited by nodes) — and the Internet texts as 
a rule do not contain restricted possibilities in the process of navigation between the links. The information gaps 
between information are for low pre-knowledge learners too wide in order for them to build inferential bridges be- 
tween them and to construct text coherence and — consequently comprehension. To sum up, previous research and 
results of the present study agree, not all students would benefit from introducing ICT supported education Pulko, 
Zemljak 2013), especially from implementing (not didactically remodeled) open hypertexts — which is a typical text 
structure on the internet. Internet text structure confronts the reader with additional cognitive load and additional 
stress, which hampers reading comprehension and the process of learning. 


And what does this mean for natural science class? 


Reading and reading comprehension is an important gateway for learning and succeeding at school. There 
are remarkable differences in off line reading: among students. According to Leu et all. (2014) American results 
show in both the 2011 and the 2013 National Assessment of Educational Progress (NAEP) for reading, a difference 
of two thirds of a standard deviation in scaled reading scores between eighth-grade students eligible for the Na- 
tional School Lunch Program and those who were not (NCES, 2011b, 2013). The difference favored economically 
advantaged students. This is roughly the difference between scores at the 25th percentile and the 50th percentile 
(see NCES, 2011b , 2013) or two to three years of schooling in the middle school and high school years (Hill, Bloom, 
Black, & Lipsey, 2008). 

This difference in reading performance had - statistically confirmed X serious consequences on learning sci- 
ence (Leu et al., 2014). Differences in science achievement had been measured: In the 2009 NAEP for science, there 
was a difference of nearly one standard deviation between eighth grade students eligible for the National School 
Lunch Program and those who were not. This represents a difference between scoring at the 20th percentile and 
the 50th percentile (NCES, 2011a) or two to three years of schooling in the middle school and high school years 
(Hill et al., 2008). 

The score difference in reading performance and its influence on learning science were X as mentioned X 
calculated for the process, when students used traditional offline — linear learning materials. But what would have 
happened, if natural science teachers would implement e-didactical materials and knowledge on the Internet for 
all students at the same time — without any differentiation? According to Leu et al. (2014) the learning achievements 
are, when reading online, even more correlated with reading comprehension results. 


Conclusions 


The new school agendas all over the world recommend Internet “text” as a knowledge source and as a source 
of information — and natural science didactics seem to be more open to the concept than social science didactics. 
But, as recent research, including presented study, reveals that at least half of digital natives have in the process of 
learning online remarkable difficulties, which will seriously, decrease their learning results, if the switch from con- 
ventional to ICT teaching paradigm will happen too soon and without differentiation according to the level of their 
online research and comprehension literacy competence. Digital natives, despite the fact that they are growing up 
in an online world and spending thousands of hours in online gaming, texting and socializing, have limited skills in 
computer basics and even more limited skills in searching for the information on the Internet, navigating websites 
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and evaluating the information they have found. Previous research, as well as the results of this study, suggests that 
instruction in online research and comprehension should be included in literacy curriculum (OECD, 2010) and that 
natural science research and comprehension competence should be included into the curricula of every natural 
science subject at all levels of the school system. 


Note 


This paper was presented at the 1* International Baltic Symposium on Science and Technology 
Education (BalticSTE2015) ‘State-of-the-Art and Future Perspectives, 15-18 June 2015, Siauliai, Lithuania. 
It was approved by the Symposium scientific committee and recommended for publication in 
Journal of Baltic Science Education. A short version of this paper is published in the symposium proceedings 
(https://www.academia.edu/13101334/STATE-OF-THE-ART_AND_FUTURE_PERSPECTIVES). 
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